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Introductory Statement 



The Center's missibn is to improve teaching in American schools. 
Its wotk is carried out through five programs: 



* Exploratory and Related Studies 

This study, which represents part of several years' work on devising 
tests to measure children's relative proficiency in two languages or two 
speech varieties, began under the Program on Teaching Students from Low- 
Income Areas and will be continued by the Program on Teaching and Lin- 
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Abstract 



A preliminary version of an oral proficiency test in English and 
Spanish for bxlingual speakers was developed. The original item pool 
used parallel English aad Spanish ^versions of a vocabulary test (91 items) 
and a grammar test (38 items). The vocabulary items were chosen to 
represent four different domains (home, neighborhood, church, school), 
the grammar items to elicit English responses likely to contain construc- 
tions subject to inters or irttralinguistic interference. 

A preliminary test using the whole item pool was administered to 45 
bilingual students (18 first graders, 17 fifth graders, 10 ninth graders) 
of whom 41 took complete versions of both tests in English and in Spanish. 
On the basis of difficulty and significant tendencies shown (dominance of 
Spanish over English in grammar and in the three domains of vocabulary 
other than school), 32 items (8 for each domain) were selected as the 
revised version of the vocabulary test and 16 items as the revised 
version of the grammar test. The tests based on ^hese items had consid- 
erable reliability (Cronbach a for Spanish vocabulary .86; for English 
vocabulary .94;- for Spanish grammar .94; for English grammar .96). 

Five independent variables (grade, sex, years of schooling in the 
United States, place of birth, and reported language use with father, 
mother, siblings, and friends) were used in a univariate analysis of 
te'st scores in order to determine whether they contributed significant- 
ly to the variance in those scores. In most instances the significant 
contribution of the independent variables seemed to be in the expected 
direction, e.g., performance in English increased with grade, years of 
schooling in the United States, and use of English with siblings. Most 
of the subjects reported speaking Spanish with their parents (mother, 36; 
father, 34). Language use with the father showed no signif icajTt relation 
to the test results. Childreri who reported using Spanish with their 
mother performed better on the neighborhood and school domains of the 
Spanish vocabulary test; those reporting use of English with their mother 
performed better on the neighborhood domain of the English vocabulary test. 

Modification of the grammar test is required in order to focus 
students' responses on the intended grammatical structures. Administra- 
tion of both parts of the proficiency test on grammar to larger and 
different samples of pupils, and the correlation of test results with 
other independent variables are needed to establish the validity o-f the 
test as an instrument for diagnosis and for the measurement of achievement. 



A PILOT STUDY CONCERNING THE DEVELOPMENT OF 
A SPANISH/ENGLISH ORAL PROFICIENCY TEST 



Robert L. Politzer and Maryann McKay 

Purpose of the Study 

The institution of bilingual education programs has necessitated the" ' 
development of tests that allow us to assess the language abilities of 
young children in English (e.g.. see Brengelman S Manning. 1964) as well 
as their relative proficiency in two languages. Several attempts have 
been made to measure the language abilities of the same individu-l in 
English and Spanish. In general, it has been easier to construct 
reliable tests measuring auditory comprehension (e.g.. Carrow. 1973; 
Cervenka. 1967). w^Sjare easily scored, than to develop tests of 

language production.-/ 

one of the most ambitious attempts to measure the language production 
of bilingualAl<»«" undertaken in conjunction with the development^ 
of the Michian/oral Language series (Michigan Oral Language Productive 
Test 1970). M discrete item production test was devised for the purpose. 
It elicits, b/a combination of pictures and verbal cues, specific fea- 
tures of standard English grammar often missed by speakers of nonstandard 
dialects and/or speakers whose dominant language is Spanish. The test s 
designed for the kindergarten and early grade levels. It requires a fairly 
elaborate scoring system because of the obvious difficulty of elicit ng 
the expected response by a combination of pictures and verbal stimuli. , 
in addition, it has been criticized for using picorial cues (such as a 
fishing pole) which may be outside the cultural experience of many chil- 
dren and thus influence the results. 

Burt and Dulay (n.d.). who are among the people who have raised 
this criticism of the Michigan test, developed a promising Bilingual 
syntax Measure of their own. Their test assesses primarily the syntacti- 
cal complexity of sentences produced by children in English and Spanish. 
It has been tried out with children from kindergarten to third grade. 
Like the Michigan Oral Language Test, the Bilingual Syntax Measure relies 
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on a combination of pictures and verbal cue^ for eliciting oral responses. 
Pictures and oral cues are chosen in order to produce certain English and 
Spanish constructions. The choice of construction is influenced by the 
sequence in which grammatical morphemes appear in first language acquisi- 
tion (see Brown, 1973). However, any appropriate response given by a 
subject is accepted and analyzed according to a formula that establishes 
a ratio* between the semantic feature used in the child's response and the 
Corresponding adult version as well as the ratio of functors used by the 
child to those that would be used by an adult who speaks standard English 
(cr Spanish). The test promises to be a useful research instrument, but, 
ae of the moment at least, the scoring system appears rather complex. 

The problem of complex scoring procedures for unpredictable responses 
also besets other bilingual language tests that combine assessment of ' 
cognitive and language development in elementary school children. Thus 
the language section of the LCT, Language-Cognition Test (Stemmler, 1967), 
relies on analyzing the subject's responses according to basic sentence 
typeis, fundamental transformations, type of verb constructions, and 
adjectival usage. 

The sociolinguistic concept of language dominance by domains (e.g., 
Fishman, 1972) was introduced recently to the testing situation. / 
Some evaluators (Cohen, forthcoming) have used children's ability to name 
words associated with different domains such as home, neighborhood, church, 
school. A test for oral proficiency which recognizes different domains 
has been used in New Mexico schools in bilingual education programs and 
was recently described by. the experimenters involved in its development 
(Spolsky, Murphy, Holm, & Ferrel, 1972). This particular test, however, 
was designed for general classification lather than precise testing and 
used a rating scale evaluation of communicative competence and self- 
reporting on the part of the students. It^'is not a scorable, discrete 
item' instrument. Such an instrument is represented by the Language Use 
(Use del Lenguaje) test produced by the Guidance Testing Associates 
(Cooperative Inter-American Tests, n.d.). However, this instrument does 
not directly test and score the production of ianguage; nor do its three 
subscorfcs (active vocabulary, expression, and total) relate in any way 
to the question of dominance by domain. 

9 
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There is. then, still a need to develop tests measuring production 
of Spanish and English which will tneet the following requirements. 

1. The_te8t should expand present efforts and measure productive 
ability n^ only in the primary grades but also at the higher 

levels. ''"^ 

2. It should be relatively easy to score. 

3. It sho"uld measure in such a way that Spanish and English 
scores are comarable. and thus be capable of demonstrating 
imbalance in f^or of either language. 

4. It should give information concerning dominance by domains. 
Seciolinguistic research has amply demonstrated that 
knowledge of vocabulary varies according to domain in 
practically every bilinguaUspeaker . A "general test" of 
vocabulary knowledge is thu4 likely to give a fragmentary 
and probably biased picture of the real language ability 
of a bilingual speaker. 

The test for this pilot study was designed to meet these four 

requirements . 

Administration of t he Item Pool 

The method used in the development of the test was to construct 
separate vocabulary and grammar tests based on an item pool larger than 
the test itself. The vocabulary item pool was based on eight pictures 
(see Appendix A). Each picture illustrated a variety of nouns and verbs, 
and subjects were asked to identify each item in response to the sample 
question: What is this (he. she)? or What is (are) he (she. they) doing? 
The same picture was used for the English and Spanish versions of the 
test. The actions, persons, or objects in each picture were chosen to 
illustrate different domains of linguistic usage. 
Home: Pictures I and II 

Neighborhood: Pictures III and IV 
Church: Pictures V and VI 

School: Pictures VII and VIII 
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Igerador (hielera) 



The s'^ecific vocabulary items illustrated and asked for by each picture 
were the following. 

Picture I n 

1. to eat; comer 

2. to cook; cocinar 

3. to feed the baby; dar de coiner (al bebe) 

4. to build a (home); hacer (construir) (una casa) 

5. to cry; llorar 

Picture II 

1. mother; mama 

2 • grandmother; abuela \ 

3. child; nino 

4. father; papa 

5. sink; lavandero (fregadero) 

6. kitchen; cocina 

7. living room; sala 

8. bedroom; cuarto (llormitorio; recamara; habitacion) 

9. chair; silla (sillon) \ 
10. table; mesa ^ 

11 • lamp; lampara 

12. carpet (rug); alfc>nd)ra 

13. refrigerator; refi 

14. to wash (dishes); lavar (los platos) 

15. to sew; coser / 

16. to sleep; dormir • 

17. to mow (cut) lawn (grass); cortar el pasto (zacate) 
Picture III 

1. girl; nina 

2. boy; nino ' 

3 . man ; s eno r (homb re ) 

4. boys; ninos 

5. sun; sol 

6. run; correr 

7. buy (ice cream); comprar (helado, nieve, paleta) 

8. sell (ice cream); vender (helado) 

9. fight (hit); pelear (golpear) 

10. play (ball); jugar (a la pel.ota) 

11. to swing; estar columpiando 

12. shine; jbrillar 

Picture IV 

1. owner; dueno 

2 . apt>le ; manzana 

3. pear; pera 

4. bread ; pan 

5. bananas; platano 

6. milk; leche 
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?• 6ggs ; huevos 

8, butcher; camicero 

9 , broom;, escoba 

10, woman (lady); mujer (senora) 

11, cash register; caja (registradora) 

12, scales ;' balanza ^ . 

13- sli^e (cut) (meat); cortar (rebajar, tajar) came 

14, sweep; barrer 

15- pay; pagar v 

16, carry (a bag) ; llevar (una bolsa) 

17, cart; carrito (cochecito) 



Picture V 



1, cross; cruz 

2, altar; altar 

3, priest; padre (cura) 

4, altar boy; acolito (monacillo) 

5, candle?; vela? 

6, kneel; hincar' (arrodillar) 

7, praying; rezat , 

8, statue; la Vitgen (santa; imagen) 

9, celebrate (say) mass; celebrar (dar) la misa 



Picture VI 



1, bride; novia 

2, groom; novio 

3, bridesmaid; damas (madrinas) 

4, veil; velo 

5, flowers; f lores (ramo) - 
6- singer (soloist); cantante 

7, organ, piano; piano (organo) 

8, to get married; casarse 

9, to marry; casa^ 



Picture VJI^ 

I, class (room); clase (cuarto; sala; salon de clase) 
2 teacher: tnaestra (profe^ora; senorita) 

3, students (pupils); ^niSos (estudiantes ; alumnos) 

4, blackboard; pizzaron (pizarra) 

5, desk; escritorio - 

6, notebook (paper); cuademo (papel) 

7, books; libros 
8- chalk; tiza 
9, flag; bandera 

10. to write; escribir 

II, to read; leer ^ i^«4^\ 

12, to sharpen (a pencil); sacar punta (un lapiz) 

13, to draw; dibujar 

14, to raise (a hand); levantar (la mano; alzar) 
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Picture VIII 

1. to sing; cantar * " 

2. to watch; mirar ' 

3. to dance; bailar 

4. to play the guitat; tocar la guitarra 

5. stage; escenario (foro; plataforma) 

6. lights; luces (focos) ~ '~ 

7. curtains; cortina , 

8. janitor; barrendero (barredor) 

The grammar test items assumed that a series of questions related to^ 
particular pictorial stimuli would elicit responses containing grammatical 
structures known to contain some production problems for speakers of 
Spanish (see Politzer fit Ramirez, 1973) • ^The Spanish version of the test 
was simply a translation of the English test. In other words, the Spanish 
version was not. based on a particular lingui-^tic item but provided a 
general w^y of comparing proficiency in grammar in the two languages. 
Specifically, the grampiatical problems and responses aimed at by the 
original '38 items developed for the grammar test were the following: 

Indefinite Article 

1. It is a book. * Es un libro. 

2. It is an apple. Es ima manzana. 

Definite Article 

3. It is the mother. Es la mama. 

4. She washes the dishes. Ella lava los^ platos. 

Third Person Singular of Verb 

5. He (the boy) eats at noon. Come al mediodia. 

6. He goes to school. Va d la escuela. 

7. He studies (reads) books. Estudia. ( Lee los libros.) 

■ t 

8. What time is it? iQue hota es? (iTi^ne la hora? 

^ iQue i^oras son?) 

P ast Tense 

9. He washed his hands. Se lavo las manos. 

10. He read books (studied). Leia . (Estudiaba los libros.) 
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11. 
12. 
13. 
14. 

15. 



16. 
17. 



18. 
19. 

20, 
21. 



Verb Agreement 
They dance (are dancing) . (Ellos > ^ " ^« - 

they watch fare watching) TV. (ElJc ■ Vi- television. 

She reada (is reading) a story. (Ella) lee cuento (Ubro). 
They fio (are ioinfi) to church. (Ellos) van a la igleaia. 

— — Passive 



The\ car was hit (smashed , 
mashed) . 



(El coche) fue pegado 
(chocado, golpeado). 



22. 
23. 

24. 
25. 
26. 

2rf. 
29. 



Negation' 

Donlt hit the boy.' • No fie^ al nine. (No le pegues.) 

(Because) she doesn't have (Porque) (ella) no tiene dinero. 

any money * \ 

Question^ ^ 
DO iou want to play? Quieres j'ugar? (iEs tu pelota?) 

(Is this your ball?) 
Did 20U break (who broke) iRomEiste (auien »2Eio) la ventana? 

the window? 

Comparative 

He (this one) is taller (E^te) es mas alto. 

(bigger ) . 

She (this one. girl) has (Esta. ella) tiene mas cosas. 

more things. 

Possessive 

The girl's . De la nlHa (muchacha, ella) . 

The mother's (iady^s SSJ^ ^a"^ ""J*'^ * 

woman ' s ) . 

Progressive Form of Present Tense 

He is swimming . ISii nadando . 

He is laughing f'^ \ ; Esti riendo . 

He is drinking . ^ ! Esta bebiendo (tomando) . 

He is sleeping . | Esta dunriendo , 

Pronouns (p pflpjlAfe^ Personal^ 



At himself . 

No, he does (pays). 



/ A si mtsatt^ . (Se mira a el. ) 
No, el (paga). 
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^ Preposition 

30. It Is on the t^^bl\^ Esta en la mesa (enclma de, 

(on top of/ sobre . 

31. In the bed (on, on top of ). En la cama (sob re, enclma).^ j 

32. .Under the bed.^ Debajo de la cama. y 

33. Above th^ blackboard Arriba de la plzarra. - i 

(on top of). ^ 

/ Verb + Verb Const rvjtl on (Subordinate Clauses ) 
% / ^ ' \ • 

34. ^e wants to open the door Qulere abrlr la puerta 
(go in, leave). (entrar, salir)> 



35. He can't find his shoe. No puede encontrar su zapato. 

36. (Because) he has to stay (Porque) tiene £ue guedarse. 

home . 

37. So that he won't miss the Para no perder el camion. 

bus. (in order to catch 
the bus) . 

38. So that tthe, rain won't Para ^ no entrara la ll^ivjfe 

come in. (el agua). 

The vocabulary tests and grapmiar tests were administered to 45 
bilingual students (18 first graders, U? fifth graders, and 10 ninth 
graders; for further breakdown see teelt analysis below). Forty-one of 
the subjects took the complete Spanish and English versions of all tests. 
The tests were administered by five university students, themselves 
bilingual Mexican-Americans.^ The examiners were carefully instructed in 
how to administer each test orally and individually and how to mark the 
answ^ sheet provided for each pupil. 

The items ih the vocabulary test were marked correct if the subject 
produced the word envisaged/ or any other acceptable or appropriate vocab- 
>^lary item that could be considered nonstandard or dialectal. ^Nonstandard 
Spanish vocabulary was accepted because the purpose of the test was to 
assess relative proficiency in Spanish and English, not to control for 
standard vs. nonstandard varieties of Spanish. A response to a vocabulary 
item was marked incorrect only if the student (a) attempted a circumlocu- 
tion of a concept (e.g., "he is making the room clean" instead of "he 
sweeps"), (b) gave a response that did not describe the action or person 
or object, or (c) gave no response at all. 
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The ex^minera were inatructjed to score responses on the grammar test 
according tp the fallowing ayatem: (a) A reaponae that was totally 
grammatically correct and contained the grammatical construction aimed at 
by the test item was marked correct by simply putting a check mark next 
to the item on the answer sheet, (b) Any response that was both 
appropriate and grammatical was accepted as correct and writlen on the 
anawer aheet by the eWner. (c) All responses that were either 
inappropriate or un.rammatical were incorrect and were Jotted down on 
the anawer aheet by the examiner. No response was counted as incotreet- 
and the examiner left the answer space %lank. 

^ Preliminary Testing and Analysis 

The preliminary analysis of Jthe tests and t*t itema was undertaken 
to determine (a) the degree of difficulty of the test and the itema. and 
(b) the degree to which the testa and the individual ite« differentiated 
knowledge of Spaniah and Engliah. Table 1 ahowa the mean scores achieved 
by all aubjecta who took both the Engliah and Spaniah veraiona of the 
vocabulary and the grammar teata. In general, the aubjecta performed 
better in Spaniah than in Engliah with the exception of the vocabulary 
teat baaed on Picture VII (achool domain), where there i- a aignificant 
difference in favor of Engliah. Significant differencea in favor of 
spaniah are ahown by vocabulary teata based t,n Pictures I (home domain). 
IV (neighborhood). V and VI (church), and the grammar teat. The results 
confirmed the expected dominance configuration. 

' The analyaia of the reaponaea to apecific itema (Tablpa 2 and 3) 
indicated that moat items were very eaay for theae aubjects. Therefore the 
itema choaen for the revised test were those which (1) showed some degree 
of difficulty, and (2) discriminated between Engliah and Spanish accord- 
ing to the tendenciea ahown by the teat as a whole, e.g.. in favor of 
Spanish for the grammar teat and the home, neighborhood, and church 
domaina of the vocabulary, and in favor of Engliah in the achool domain 

of the vocabulary test. ^ 

An inspection of scores on the preliminary test for individual 
vocabulary itema showed that in most cases ^n which vocabulary ite 



TABLE 1 

Means, Standard Deviations, and Significant Difference of Means 
on Preliminary Vocabulary Tests and Grammar Test (N « 41) 



Test 



Picture VIII^ 



Difference T-Valu^. 
Mean S,D, of Mean (2-Tail Prbb) 



Picture I 

Spanish 4.37 0.73 5^ -2.95** 

English 3^83 1.18 

Picture II 

Spanish 14.49 1.87 _q ^2 -0.30 

English 14.37, 2.51 

Picture III 

Spanish -^iOvSa . 1.47 _q 57 

English '10.39 1.80 

Picture IV 

Spanish 13.54 2.22 ^2.75** 

English 'itl.92 3.02 

Picture V 

Spanish 6.22 1.79 -2.00 -5.19** 

English. 4.22 r~ 2.08 

Picture VI 

Spanish 6.32 1.82 -1 39 ' -3.40** 

English 4.93 2.04 

Picture VII* ' , 

Spanish 11.28 2.70 ^ j £.49* 

English 12.56 2.11 ' / 



^ / 



Spanish 7.38 9.72 _qj^^ ' / _o.40 

English 6.75 2.09 J 

Grammar Test ' i 

Spanish 35.59 2.33 _2 22 / -2.47** 

. English 33.37 Cj'^^ ^ 



^ - 40 
*p < .05 
**p < .01 
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TABLE 2 

Correct Responses and Balance Score (Spanish-English) 
for Items on Preliminary Vocabulary Test 




Picture 

1. 
2. 
3. 

4.. 
5. 

Picture II 

1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
11. 
12. 
13. 
14. 
15. 
16. 
17. 

Picture III 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 
10. 
11. 
12. 



40 


40 


0 


34 


30 


+ 4 


39 


35 


+ 4 


41 


27 


+14 


40 


39 


+ 1 


39 


38 


+ 1 


38 


40 


- 2 


28 


22 


+ 6 


41 


35 


+ 6 


15 


32 


-17 


41 


35 


+ 6 


34 


28 


+ 6 


36 


30 


+ 6 


35 


32 


+ 3 


41 


41 


0 


39 


38 


+ 1 


34 


36 


- 2 


40 


38 


+ 2 


31 


32 


7- 1 


35 


32 


4 3 


35 


39 


; - 4 


37 


37 





42. 


40 


+ 2 


40 


41 


- 1 


38 


41 


- 3 


40 


39 


- 1 


39 


41 


- 2 


39 


34 


+ 5 


38 


36 


+ 2 


25 


24 


+ 1 


39 


37 


+ 2 


38 


40 


- 2 


34 


34 


0 


25 


26 


- 1 



1. 

2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
11. 
12. 
13. 
14. 
15. 
16. 
17. 

Picture V 

1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 

Picture VI 

1. 
2. 
3. 
4. 

/■ 5. 
6. 
7. 
8. 
9. 



19 


19 


0 


40 


40 


0 


41 


38 


+ 3 


40 


37 


+ 3 


36 


41 


- 5 


40 


41 


- 1 


41 


41 


0 


24 


19 


+ 5 


39 


35 


+ 4 


35 


36 


- 1 


10 


16 


- 6 


. 15 


7 


+ 8 


39 


34 


+ 5 


30 


24 


6 


31 


23 


+ 8 


38 


33 


+ 5 


34 


27 


+ 7 



37 

11 
38 
11 
34 
36 
32 
27 
30 



34 
•4 
17 

> 
35 
31 
29 

8 
10 



+ 3 . 
+ 7 
+21 
-t- 6 
- 1 
+ 5 
+ 3 
+19 
+20 



34 


20 


+14 


33 


13 


+20 


19 


4 


+15 


18 


11 


+ 7 


35 


37 


- 2 


19 


30 


-11 


37 


39 


- 2 


39 


27 


+12 • 


37 


27 


+10 
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TABLE 2 (continued) 



- 

Test 


Spanish 


English 


Balance 


lesc 


Spanish 


Pn ar1 4 fiVi 


An r& 

UCI'L Oil 


ITT T 

Picture VII 






i 


t 

IT XCLli&C 


VTTT 




/ 

+ 2 / 


1. 


39 


35 


1. 


40 


38 


2. 


40 


41 


- 1 


2. 


29 


' 22 


+ 7/ 


3. 


40 


41 


- 1 


3. 


41 


\ 38 






33 


38 


- 5 . 


4. 


31 


34 




\ 5- 


27 


35 


- 8 


5. 


6 


• 32 


^ 6. 


37 


38 


- 1, 


6. 


4Q 


40 


1 0 


7. ' 


40 


41 


- 1 


7. 


12 


35 


A 5 


8. 


16 


38 


-22 


8. 


23 


/ -11 


\ 9. 


40 


38 


+ 2 










^10. 


34 


40 


- 6 






/ 






39 


40 


- 1 






/' 






26 


40 


-14 










13, 


26 


31 


5 






/' 




14 


37 


37 


0 








/ 
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TABLE 3 



Correct Responses and Balance Score (Spanish-English) 
for Items on the Preliminary Grammar Test • 



Spanish 



English, 



Balance 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26. 

27 
28 
29 
30 
31 
^ 32 
33 
34 
35 
36 
37 
38 



42 
41 
42 

42 

40 

40 

39 

41 

38 

34 

41 

40 

38 

42 

37 

41 

41 

41 

36 

41 

40 

42 

42 
42 
42 
42 
41 
39 

.42 
42 
42 
42 
40 
41 
42 
42 
39 
42 



42 
41 ♦ 
42 
38 
37 
35 
39 
40 
30 
35 
39 
38 
37 
37 
32 
37 
31 
35 
34 
32 
38 
37 
39 
40 
41 
41 
42 
35 
40 
41 
I 42 
39 
36 
37 
34 
37 
33 
36 



0 
0 
0 

+ 4\ 
+ 3 
+ 5 
0 

1 



+ 
+ 

+ 
+ 
+ 
+ 
+ 
+ 
+10 
+ 6 
+ 2 
+ 9 
+ 
+ 
+ 
+ 
• + 
+ 



+ 
+ 
+ 

+ 
+ 
+ 
+ 
+ 
+ 



2 
5 
3 
2 
1 
1 
1 
4 
2. 
1 
0 
3 
4 
4 
8 
5 
6 



\ 



+ 6 



\ 
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not follow the general pattern indicated by the domain, the vocabulary 
item was not unambiguously associated with that domain. For example, in 
Picture VIII, items which showed dominance of Spanish over English within 
the school domain were: 1* sing, 2, watch, 3. dance, 7. curtains. In 
Picture VI (church domain), the items which showed some dominance of 
English are less directly related to the church setting than other items 
in the picture (5, flowers, 6, singer, 7. organ, piano). Finally, a few 
items were excluded because scoring had been ambiguous or erroneous. 
Thus item 3 on Picture V seems to be reasonably difficult in English and 
to discriminate well between Spanish and English, However, the low 
English score is due entirely to the rejection of the acceptable response 
father (for priest) in the initial scoring. Two final considerations 
influencing the choice of items for the test were to have an equal 
number of test items for each domain of the vocabulary and to produce a 
test that could be administered in a reasonably short time (about 40 
minutes for the total test in both languages). 

The test items chosen for the final testing were these: 

Vocabulary Tests 

1, Domain of Home 

Picture. I: Items 2, 3, 4 
Picture II: Items 6, 7, 8, 9, 15 

2, Domain of Neighborhood 

Picture III: None 

Picture IV: Items 8, 9, 12, 13, 14, 15, 16, .17 

3, Domain of Church 

Picture V: Items 2, 8, 9 
Picture VI: Items 1, 2, 3, 4, 8 

4, Domain pf School 

Pictu^ VII: Items 4, 5, 8, 10, 12, 13 
Picture VIII: Items 5, 8 

Grammar Test j 
\ Items 6, 9, 14, 15, 16, 17, 18, 20, 22, 28, 33, 34, 35, 36, 37, 38, 

\ Scores on these items were reanalyzed, and the -results are given 

below. 
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Analysis of Revised Test 

Reliability 

Reliability was determined for the vocabulary tests (32 Items each) 



as a 



whole and for the two grammar tests (see Table 4). 



TABLE 4 
Cronbach a Reliability 



Test 9t 

Spanish Vocabulary 0.86 

English Vocabulary 0.94 

Spanish Grammar 0.94 

English Grammar 0.96 



Dominance by Domain 

Since the test items had been selected on the basis of distinguishing 
dondnance configurations shown by the larger preliminary iuem pools, the 
final test reflected, of course, the dominance differences indicated by 
the preliminary analysis. As can be seen from Table 5. three of the four 
sections of the vocabulary test (school is the exception) and the grammar 
test show significant dominance of Spanish over English (p < .01). 

Variances in Test Results 

The investigators felt that it would be of interest to determine 
whether variance in performance on the tes. was significantly influenced 
by certain factors. Information concerning some of these (grade, sex, 
years of schooling in USA, place of birth) was easily available. In 
addition, the subjects had been asked at the interview whether they used 
English. Spanish, or both languages with either of their parents, their 
friends, or their siblings. 
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TABLE 5 

Means, Standard Deviations, and Significant Difference of 
Means for 4 Vocabulary Subtests and Grammar Test (N * 41) 



Domain 


Mead 


S.D. 


Difference 
of Mean 


1— Value 
(2-Tail Prob) 


Home 


< 








Test la Soanish 


6.68 


1.11 


1.12 


3.52 * 


Test lb English 


5.56 


2.03 






Neighborhood 










Test 2a Spanish 


6.02 


l.53tk 


1.46 




Test 2b English 


4.56 


2.13 






Church 










Test 3a Spanish 


5.07 


1.99 


2.66 


6.64** 


Test 3b English 


2.41 


2.04 






School 










Test 4a Spanish 


4.24 


1.91 


-2.24 


-4.05** 


Test 4b English 


6.49 


2.09 






Grammar Test 










Spanish 


15.63 


0.77 


1.12 


2.72** 


English 


14.51 


2.64 




1 



V **p < .01 



Since the sample of pupils was small (only the 41 students who had 
completed all portions of both tests were included in the analysis), only 
a univariate analysis of variance was undertaken to determine to what 
extent each of these independent variables influenced the test scores. 
Tables 6 to 10 summarize the findings of this analysis. A significant 
(p < .05) variance due to the particular independent variable for that 
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table is indicated by a box around the means and standard deviations 
reported in the tables. The analysis of variance tables reporting the 
F ratios on which the findings of significance are based are shown in the 
Appendix B. Only the analysis of variance tables showing significant 

variance are reported. 

Influence of grade . Grade in school influenced performance on the 
Spanish vocabulary subtests 1 (home). 3 (church), and 4 (school) with 
performance obviously increasing from grade to grade as is shown in 
Table's. On the English tests, grade contributed significantly to 
variance except on subtest 2 (neighborhood). However, it should be 
noted that on all English 'tests the performance of the ninth graders 
was. in fact, lower than that of the fifth graders. The number of 
ninth graders tested was only seven, and two were recent arrivals from. 
Mexico. 

The difference between the means of fifth and ninth graders in the 
English gramar test is significant only at the 0.1 level: T-value 1.77. 

degrees of freedom 6.14. 

Year, of schooliPfi rhe United States . Most of the. children tested 
in this study were not bom in the United States (see Table 8). (Even for 
those bbrn in the United States the ties with- Mexican culture were very 
strong.) None of the ninth graders had all of their schooling in the 
united «tatjes. Table 7 shows that years of schooling within the United 
, States Scted performance on two of the English vocabulary subtests 
(h^tliwchurcb) and on 'the grammar test in the expected direction: 
iW^formance increased in proportion to schooling in the United 
States. Influence of schooling in the United States on the English 
grammar test barely reached significance (p < .05). However, we must 
keep in mind that total variance on the grammar test was rather small. 

Place of birth . Tdble 8 shows that the only test signif f cantly 
affected by place^^irth (U.S.A. vs. Mexico) was the Spanisii vocabulary 
subtest 3a (church) "in which^-not unexpectedly-those »ho were bom in 
Mexico performed better t1,an 'subjects bom in the United States. 

Sex. None of the test scores showed significant variance due to sex 
of subjects (see Table 9). . . 

r 
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I 

Reported language use . Most of the 41 subjects who took all the 
tests reported that they spoke English with their parents (36 with the 
mother, 34 with the father). The chances of finding significant variance 
i due> to different language use with parents were therefore rather small. 
Table 10 shows that language use with the father shewed in fact no signifi- 
cant relation to the test^results. Language use with the mother did con- 
tribute significantly to variance in threp instances and in the expected 
direction. Those speaking Spanish with their mothers did better on the 
Spanish Vocabulary subtests 2 (neighborhood) anH 4 (school). Those 
speaking English did better on. the English vocabulary subtest 2 (neigh- 
borhood) . 

Language spoken with siblings evidently had no influence on the 
performance on the Spanish tests, but it did contribute to significant 
variance in all but one (vocabulary test 3, church) of the English tests, 
and again in the expected di'-ection. Those who spoke Fnglish with their 
siblings scored higher on the English tests. 

Reported language use with friends shows^ that those who spoke 
Spanish with their friends had ^ Twer scores on two English vocabulary 
subtests, 2 (neighborhood) and 4 (school) — a finding which is certainly 
not unexpected. A bit puzzling is the finding that a small but signifi- 
cant variance on the Spanish grammar test appears to be related to reported 
language us,e with friends. Those who reported speaking English ££ Spanish 
with friends received better scores than th^oSe who reported speaking both , 
languages. The highest Spanish grammar test score was achieved by tlfose 
who reported speaking English with their friends! ' In view of the vagueness 
of the questions asked of the subjects (e.g., whom did they perceive as 
their "friends") and the small variance which appeals on the Spanish 
grammar test as a whole, this f in( ing certainly should not be over interpreted. 
If the use of both Spanish and English in everyday life influences Spanish 
proficiency, why was reported language use with siblings not a significant 
variable? More research using a larger sample showing more variance in 
Spanish is necessary to answer this question and to confirm these pre- 
liminary findings. 
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Conclusion 



The test discussed in this report is obviously in need of further 
refinement and should be subjected to fyrther experimentation. While in 
practically all cases it was relatively eas/to score a response as either 
correct or incorrect (i.e. inappropriate or containing an error). Table 11 
(responses on the grammar test) indicates quite clearly that many test 
items did not elicit the expected grammatical patterns. If the grammar 
test is to be maintained in the present form, as a discrete item test 
based on areas of expected inter- or intralingual interference, then a 



I 

TABLE 11 



Distribution of Responses for Items on Revised Grammar Test 



Spanish Test EnHlish Test 



Item 




b 


c 


d 


6 


38 


2 


1 . 


1 


9 


28 


10 


4 




14 


29 


13 






15 


2 


35 


5 




16 


34 


- 7 


1 




17 


33 


8 


I 




18 


38 


3 


1 




20 


28 


13 


1 




22 


35 


7 






28 


2« 


11 


2 


1 


33 


24 


16 


1 


1 


34 


/ 


5 








/ 20 


22 






36 / 


29 


13 






37 / 


14 


25 


3 




38 / 


11 


31 







a 


b 


c 


d 


33 


2 




2 


28 


2 


10 


2 


35 


2 . 


4 


1 


6 


26 


7 


3 


32 


5 


2 


3 


20 


11 


9 


2 


31 


4 


6 


1 


21 


11 


5 


5 


35 


2 


5 




31 


4 


3 


2 


21 


15 


4 


2 


34 


3 


1 


1 


li 


15 


8 




31 


6 


3 


2 


• 7 


26 


5 


3 


8 


29 


3 


2 



Note: Responses were counted for all students (H-41) who completed 
both the Spanish and English sections of the grammar test, 
a « correct (expected response) 
b - correct (not expected response) 
c « incorrect (inappropriate response) 
d « no response 
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way must be found to focus student response on the expected grammatical 
pattern. Perhaps it would be possible to do this by supplying at least 
the initial word of expected answers. For example, grammar test item 38: 



Even the decision on what to accept as correct in grammar and ^ 
especially in vocabulary can at times be difficult to make. It was 
explained earlier that the test does not intend to judge dialectal (e.g., 
Mexican-American) Spanish as "incorrect." However, at what point does 
borrowing from English cease to become "acceptable Mexican-American 
Spanish?" Even the possible guideline that speech should be accepted as 
dialectal (Mexican-American) if it is used in the community in which the 
pupil lives is not as clear and unambiguous as it first appears. Borrow- 
ings from English used by Spanish-speaking children may be innovations 
in their dialect — who is to say that they are "errors"? Maquina de 
lavar, washing machine, rather than lavadora , can now be considered 
regular Mexican-American Spanish, but at one time it must have been an 
innovation. Perhaps a better way to define "correctness" would be to 
accept any kind of grammatical performance or vocabulary item that 
follows the grammatical rules and word formation procedures of the 
subject's dialect. Such a decision would probably necessitate providing 
test scorers with a range or choice of acceptable correct answers. 

Finally, additional studies concerning the validity of the tests 
reported here (and of the Spanish/English balance measurement for which 
they can be used) are needed so that the test can serve as a useful 
instrument for diagnostic purposes and for measuring language achievement. 



Why does the ^rl open the window? So_ that 
^ Porque obre la vent ana ? Para que . • . . 



. . . . 
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APPENDIX B: ANALYSIS OF VARIANCE TABLES 
TABLE B-1 



Significant Contributions to Variance Vy Grade 



f 



- 


Sum of squares 




/ 

/ 

Mean square 




Test la 


\ 
\ 




Between Groups 
Within Groups 




(2) 
(38) 


■4 . 54 
1.05 


Total 


48. d7 


(40) 




F = 4.34* 


Test 3a 






Between Groups 
Within Groups 


64.85 ^ 
93.93 


(2) ^ 
(38) 


32.47 
2.47 


Total 


158.78 


(40) 




F = 13.12** 


To Q t" ^ A 

lea *TCi 






Between Groups 
Within Groups 


30.41 
115.15 


(38) 


15.20 
3.03 


Total 


145.56 


(40) 




F = 5.02* 


lest ID 






Between Groups 
Within Groups 


81.17 
112.93 


(2) 
(38) 


40.58 
2.97 


Total 


194.10 


(40) 




F - 13.66** 


Test 3b 






Between Groups 
Witbin Groups 


42.30 
123.65 


(2) 
(38) 


21.15 
3.25 


Total 


165. 9S 


(40) 





F = 6.50 
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Between Groups 
Within Groups 

Total 
F - 5.10* 



Between Groups 
Within Groups 

Total 

P - 4.76* 



TABLE B-1 (Continued) 



Sum of squares 



d£ 



Test 4b 



36.88 
137.37 

174.25 



(2) 
(38) 

(40) 



Grammar Test (English) 



55.70 
222.55 

278.25 



(2) 
(38) 

(40) 



Mean square 



18.44 
3.61 



27.85 
5.86 



*p < .05 
**p < .01 
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TABLE B-2 



Significant Contribution^ to Variance 
by. Years of School^g in USA 



/ Sufi, of squares df Mean square 



/' T est lb - . \ 

Between Groups/ 104.75 (7) ' 14.96 

Within Groups' 70.75 (30) 2.36 

Toj^ll 175.50 (37) 
, F = 6.35^* 

Test 3b 

Between Groups 75.30 (7) 10.76 

Witljin Groups 60.07 (30) 2.00 

Total 135.37 (37) 



5.37' 



Test 4b 



Between Groups 94.43 (7) 13.49 

Within Groups 173.07 (30) 5.77 

Total 267.50 (37) 



F = 2.24* 



*p < .05 
**p < .01 



TABLE B-3 

Significant Contributions to Variance by Place of Birth 



Test 3a 

Sum of squares df Mean square 



Between Groups 29.40 (1) 29.40 

Within Groups 112.38 • (38) 2.95 

Total 141.78 (39) 



F = 9.94** 



** 



p < .01 
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TABLE B-4 

Sign -f leant Contribut as to Variance 
oy Language Spoken with Mother 



I Between Groisps 
Within Groups 

Total 
F - 4-40* 



Between Groups 
Within Groups 

Total 
F - 4.05* 



\ 



Between Groups 
Within Groups 

Total 

F - 4.90* 



Sum of s quares 

Test 2a 

17.48 
75.50 

92.98 



Test 4a 



25.59 
119.97 

145.56 



Test 2b 



37.35 
144.75 

182.10 



(2) 
(38) 

(40) 



(2) 
(38) 

(40) 



(2) 
(38) 

(40) 



8.74 
1.99 



12.79 
3.16 



18.67 
3.81 



*p < .05 
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TABLE B-5 



Significant Contributions to Variance 
by Language Spoken by Siblings 



Between Groups 
Within Groups 



Total 



F =■ 4.76 



Sum of squares 



Test lb 



54.06 
140.04 

194.10 



df 



(3) 
(37) 

(40) 



Mean square 



18.02 
3.78 



Between Groups 
Within Groups 



Total 



F = 4.24* 



Between Groups 
Within Groups 



Total 



F = 4.37 



Between Groups 
Within Groups 



Total 



F = 2.86* 



*p < .05 
**p < .01 



Test 2b 



46.56 
135.54 

182.10 



Test 4b 



45.59 
128.65 

174.24 



(3) 
(37) 

(40) 



(3) 
(37) 

(40) 



Granrmar Test (English) 



52.40 
225.85 

278.25 



(3) 
(37) 

(40) 



15.52 
3.67 



15.20 
3.48 



17.47 
6.10 
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TABLE B-6 

Significant Contributions to Variance 
by Language Spoken with Friends 



Between Groups 
Within Groups 

Total 



Grammar Test (Spanish) 

4.42 (2) 
19.09 (38) 



Mean square 



Between Groups 
Within Groups 

Total 

F - 3.28* 



Between Groups 
Within Groups 

Total 

F - 3.49* 
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